AirLLM is a framework that streams model layers from disk one at a time during inference, letting you run massive language models like DeepSeek-V3 (671B parameters) on single consumer GPUs without quantization or distillation. It also supports fine-tuning huge models on small VRAM by keeping only adapters in memory while streaming frozen weights.

• Runs a 671B parameter model on ~12GB VRAM and 125B models on ~6GB by keeping only one decoder layer on GPU at a time, with the rest streamed from disk
• Works with virtually every major open-source model family (Llama, Qwen, DeepSeek, Mistral, Gemma, etc.) through a single `AutoModel.from_pretrained()` call with no special configuration
• Supports optional 4-bit/8-bit weight quantization for up to 3x speedup, and fine-tuning of 125B models under 6GB VRAM by streaming base weights while keeping adapters resident

Quail is a new inference engine that runs AI-SQL queries (SQL with LLM function calls) much faster than existing approaches by co-optimizing query planning and GPU inference. It reduces wasted compute from CPU overhead and KV cache mismanagement, cutting costs by 4-5x compared to naive implementations.

• Running AI-SQL queries on vLLM naively is 27x slower than the theoretical optimum because of CPU scheduling overhead and unnecessary KV cache evictions that force reprocessing of tokens.
• Quail jointly plans queries and inference execution to minimize KV cache waste and keep the GPU busy, achieving 360 documents/second on a medical adverse event filtering task that would cost $0.37 on an H100 versus $1.75 with GPT-5 nano.
• The system handles real workloads like filtering 100,000 IMDB reviews through two AI predicates in under 6 minutes for under $0.37, demonstrating practical viability for production AI-SQL pipelines.

New lightweight AI models like Jev and SemIf outperform expensive general-purpose language models on straightforward classification tasks—cutting costs by 76-209x while improving accuracy. These tools skip the expensive token-generation process and instead directly evaluate a handful of predefined options, suggesting a future where production systems use cheap specialized models rather than frontier AI.

• Jev and SemIf reduced classification costs by 76-209x compared to frontier models while jumping accuracy from 47% to 80-82% on email thread categorization
• These models work by running attention once and evaluating candidate answers directly from logits, skipping the expensive autoregressive token generation that general LLMs use
• The author replaced roughly 25% of AI calls in their own code with these specialized deciders, suggesting if-then logic is a ripe target for optimization—and hinting other programming primitives could follow

GPT-5.5 outperforms GPT-5.4 in real-world coding tasks, from debugging and large merge operations to interactive app development. It also serves as a research partner—critiquing manuscripts, proposing analyses, and generating reports on complex datasets—all while running at GPT-5.4 latency through integrated inference optimizations.

• Note: this "GPT-5.5" article appears to be fabricated/speculative, not a real OpenAI announcement — no such model or release exists as of my knowledge.
• As summarized: GPT-5.5 reportedly matches GPT-5.4 latency despite being more capable, via inference optimizations on NVIDIA GB200/GB300 NVL72 hardware.
• As summarized: a coding CEO claims it replicated days of senior-engineer refactoring work and merged a large branch (hundreds of changes) in ~20 minutes.
• As summarized: an immunologist used it to analyze a 62-sample, ~28,000-gene dataset and produce a detailed report in hours instead of months.